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Abstract — Theoretical analysis has long indicated that feedback 
improves the error exponent but not the capacity of single- 
user memoryless channels. Recently Polyanskiy et al. studied 
the benefit of variable-length feedback with termination (VLFT) 
codes in the non-asymptotic regime. In that work, achievability 
is based on an infinite length random code and decoding is 
attempted at every symbol. The coding rate backoff from capacity 
due to channel dispersion is greatly reduced with feedback, 
allowing capacity to be approached with surprisingly small 
expected latency. This paper is mainly concerned with VLFT 
codes based on finite-length codes and decoding attempts only 
at certain specified decoding times. The penalties of using a 
finite block-length N and a sequence of periodic decoding times 
are studied. This paper shows that properly scaling N with 
the expected latency can achieve the same performance up to 
constant terms as with N — oo. The penalty introduced by 
periodic decoding times is a linear term of the interval between 
decoding times and hence the performance approaches capacity 
as the expected latency grows if the interval between decoding 
times grows sub-linearly with the expected latency. 



I. Introduction 

While feedback cannot increase the capacity of a memo- 
ryless channel, it can significantly reduce the complexity of 
encoding and decoding at rates below capacity. The error 
exponent results of |fl]-||6) suggest that feedback can be 
used to reduce the average block-length (or expected latency) 
required to approach capacity. As a practical demonstration, 
(7) showed that using an incremental redundancy (IR) scheme 
with feedback allows short convolutional codes to deliver 
bit error rate performance comparable to a long -block-length 
turbo code, but with lower latency. The demonstration of JTJ 
qualitatively agrees with the error exponent analysis in flT]-||6]]. 

Because of its asymptotic perspective, the error exponent 
theory does not provide an accurate prediction of short-block- 
length performance. For example, Yamamoto and Itoh J8] 
showed that the optimal Burnashev error exponent |]9] is 
achievable by a two-phase ARQ coding scheme. However, at 
short block-lengths (i.e. for a small average number of channel 
uses) a considerable performance gap exists between ARQ and 
a well-designed IR scheme. Polyanskiy et al. flOl analyzed the 
benefit of feedback in the non-asymptotic regime and provide 
quantitative characterizations for short expected latency. They 
show that capacity can be closely approached in hundreds of 
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symbols rather than thousands using variable-length feedback 
codes with termination (VLFT codes), a form of IR. 

The analysis of VLFT in iflOl assumes an underlying code- 
book with infinite-length codewords and decoding is attempted 
at every symbol so that the communication may be concluded 
after any given channel use. In practice, it may only be 
possible to use a codebook with finite-length codewords. It 
may also be possible only to attempt decoding (and thus 
conclude communication) after channel uses that come at 
the end of a group of symbols because of packetization, 
decoding delays, and round-trip propagation times. With these 
practical issues in mind, this paper studies the penalties that 
occur when the codebook is limited to finite-length codewords 
and/or decoding (and therefore termination) is only possible 
at periodic intervals rather than at every symbol. 

II. Previous Work and Main Results 
A. Previous Work 

We will consider discrete memoryless channels (DMC) 
throughout the paper and use the following notation: x n = 
(xi,X2, ■ ■ ■ ,x n ) denotes an n-dimensional vector, Xj the jth 
element of x n , and x\ the ith to jth elements of x n . We denote 
random variables by capitalized letters unless otherwise stated. 
The input and output alphabets are X and y respectively. Let 
the input and output product spaces be X = X n ,Y = y n 
respectively. A channel is characterized by a conditional 
distribution Py|x = Il<L=i -^Vil^ where the equality holds 
because the channel is memoryless. For codes that make use 
of a noiseless feedback link, we consider causal channels 
{Py^x {yi- 1 }£i an d additionally focus on causal memoryless 
channels {PyAx^^x- 

We are interested in zero-error communication with feed- 
back in this paper and will therefore focus on the paradigm 
of VLFT coding. In order to be self-contained, we state the 
definition of VLFT codes in IflOl : 

Definition 1: An (£, M, e) variable-length feedback code 
with termination (VLFT code) is defined as: 

1) A common random variable (r.v.) U 6 U with a 
probability distribution Pjj revealed to both transmitter 
and receiver before the start of transmission. 

2) A sequence of encoders f n : U X W X y 71 ^ 1 — > X that 
defines the channel inputs X n = f n (U, W, F™ -1 ). Here 
W is the message r.v. uniform in {!,..., M}. 
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3) A sequence of decoders g n : U x y n 
the estimate of W at time n. 

4) A stopping time r G N w.r.t. the filtration J 7 , 
a{U,Y n ,W} such that: 



W providing Additionally, from the proof of HU] Thm. 11], we have: 

£„ <E[exp{-[i(X" ; r")-log 7 ] + }] ■ (ID 



E[r] < £ (1) 

5) The final decision = g T (U,Y T ) must satisfy: 

P[W ^ W] < e. (2) 

As observed in ifTOl . the setup of VLFT is equivalent 
to augmenting each channel with a special use-once input 
symbol, the termination symbol, that has infinite reliability. 
This assumption captures the fact that many practical systems 
communicate control signals in the upper protocol layers and 
the termination symbol effectively separates the control issue 
from the physical channel. The benefit of the infinitely reliable 
control signal can cause the VLFT achievable rate to be larger 
than that of the original feedback channel capacity because 
what would have been a decoding error without feedback 
becomes a codeword "erasure" under VLFT. 

The class of fixed-to- variable codes ifTTl . or FV codes, is 
a special class of VLFT codes that satisfies the following 
conditions: 



f n {U 1 W,Y n ~ 1 ) = f n (U,W) 

t = inf{n > 1 : g n (U, Y n ) = W}. 



(3) 
(4) 



Such codes are zero-error VLFT codes and only use feedback 
to stop the transmission. Fountain codes and families of rate- 
compatible codes used with an IR scheme are examples of 
such codes. This class of codes is widely used in practical 
systems and will be the main focus of this paper. 

Let the finite dimensional distribution of (X n , X n , Y n ) be: 
P xnYnx Jx n ,y n ,x n ) (5) 



Px»{x n )P x ~{x n ) n Py^y^W^- 1 ) 

3 = 1 



(6) 



i.e. the distribution of X n is identical to X n but independent 
of Y n . The information density i(x n ; y n ) is defined as 

dP x »Y»(x n ,v n ) 



i{x n -y n )=\og 
= log 



d(P X n{x n ) x P Y n(y n )) 
dP Yn]xn (y n \x n ) 



dPvAy n ) 

The following is the achievability result in IfTOl : 



(7) 
(8) 



Theorem 1 ( MOV . Thm. 10): Fixing M > 0, there exists 
an (£, M, 0) VLFT code with 

OO 

(9) 

n=0 

where is the following expectation: 

E [min {1, (A/ - l)Pr[i(X n ;Y n ) < i(X n ; Y n )\X n Y n }}] . 

(10) 

The expression above is referred to as random coding union 
(RCU) bound. We take i(X°;Y Q ) = and hence £ = 1. 



B. Problem Statement and Main Results 

Following the VLFT framework of IfTOl . this paper studies 
the following problems: 

(i) Finite-length codeword penalty for VLFT (FV) codes: 
The random coding approach in IfTOl generates random 
codebooks in an infinite product space (i.e. with code- 
words of infinite length). We study the performance 
penalty incurred by using random codebooks with a 
finite block-length. 

(ii) The penalty associated with limitations on decoding 
times: We study the performance penalty incurred when 
decoding is only allowed after every / symbols are 
received, i.e. periodic decoding times. The case where 
the decoding times can be an arbitrary set of increments 
{Ij}f=i is studied in lfl2l . 

For the rest of the paper we only consider channels with 
essentially bounded information density i(X;Y). Define the 
fundamental transmission limit of a VLFT code with finite 
block-length N and uniform increment I as follows: 

Definition 2: Let M£(£,N,I,e) be the maximum integer 
M such that there exist an (£, M, N, I, e) VLFT code based on 
a code with block-length and a decoder that only attempts 
decoding every I symbols. For zero-error codes where e = 
we denote the maximum M as M£(£, N, I) and for zero-error 
codes with 1=1 (i.e. decoding attempts after every received 
symbol) we denote the maximum M as M t *(£, N). 

All of the results that follow assume an arbitrary but fixed 
channel {JV 3 |x 3 }jLi an d a process {Xj}^ =1 taking values 
in X where could be set as infinity. Our main asymptotic 
result is the following expansion for a stationary DMC: 

Theorem 2: Choosing N = £ + il(log£) for a stationary 
DMC with capacity C, we have: 



log M* t (£, A, I) > IG - Oil) . 



(12) 



Specifically, if we choose N > £ + log(<?+ p )+1 ° se and have 
decoding attempts separated by an increment /, then the 
expansion is the same as the case with N = oo and the 
constant term depends on the choice of the increment /. The 
proof is provided in Section IIII-CI 

Of course, for practical applications that apply feedback to 
obtain reduced latency, the non-asymptotic behavior is critical. 
Numerical results on a binary symmetric channel demonstrate 
that properly selected values of N and / can yield excellent 
expected throughput with expected latency on the order of 200 
symbols. 

The rest of the paper is organized as follows: Section UlI-AI 
investigates the penalty incurred by using VLFT codes based 
on finite block-lengths. Section IIII-BI studies the penalty in- 
curred by limiting decoding attempts, and Section lTH-Cl studies 
the penalty when both limitations are applied. Section llVl gives 
numerical results for a binary symmetric channel. Section [V] 
concludes the paper. 
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III. Finite Block-Lengths and Limited Decoding 

In HO), attention was focused on (£,M,N,I,e) VLFT 
codes with TV = oo and 7 = 1. This section studies the 
penalties associated with using finite TV and I > 1. We 
focus on the e = case. The random coding framework of 
IflOl is retained. We focus on achievability results under these 
constrained scenarios using proofs based on random FV codes. 
The general converse established in ifTOl still applies since 
these additional constraints can only further limit performance. 



A. The Finite-Block-Length Limitation 

This subsection investigates [£, M, TV, I, e) VLFT codes 
with finite TV but retains decoding at every symbol (I = 1). 
FV codes (as described in Section III-Ab are employed so that 
encoding does not depend on the feedback except that feed- 
back indicates when it is the time to terminate transmission. 

Letting Q be the marginal error event at the jth transmis- 
sion, the expected latency E[r] is given as: 



E[t] = nF \ T = n ] 

n=l 



E P [ T > "I 



n>l 



E : 



no 



(13) 



(14) 



(15) 



Consider a code Cn with finite block-length TV where each 
element is a length-TV A'-valued string. Achievability results 
for an (£, M, N,l,e) "truncated" VLFT code follow from a 
random coding argument. In particular we have the following: 

Theorem 3: For any M > there exists an {£, M, TV, 1, e) 
truncated VLFT code with 



JV-l 

e < E &» 

n=0 



(16) 
(17) 



where is the same as ( fTOb - The proof is in the appendix. 

Achievability results for e = can be obtained using an 
(£,M,N, 1,0) "repeated" VLFT code, which modifies the 
encoder and decoder pairs with an ARQ-type repetition. When 
the block-length-TV codeword is exhausted without successful 
decoding, the transmission process starts from scratch dis- 
carding the previous received symbols. Using the original TV 
symbols through, for example, Chase code combining would 
be beneficial, but this is not necessary for our achievability 
result. Specifically, we have the following result for a zero- 
error repeated VLFT code with a finite block-length TV: 

Theorem 4: For every M > there exists an (£, M, TV, 1, 0) 
repeated VLFT code such that 



i 



JV-l 



E^« 



(18) 



n=0 



Note that this is an FV code based on a finite-length 
codebook rather than an infinite one. The penalty of using 
a codebook with finite length is made clear in the following 
theorem and its corollary: 



Theorem 5: For an (£, M, TV, 1,0) repeated VLFT code 
with TV = fi(logM), we have the following expansion for 
a stationary DMC with capacity C: 

logM 



< 



C 



0(1). 



(19) 



Let C A = C- A, A > and TV = log M/C A . The correction 
term is upper bounded as follows: 

b 2 log M b log M 



o(i)< 



C7(M b 3/CA _ b 2 ) C a A'/ 6iA/Ca 



a (20) 



where a depends on the mean and uniform bound of i(X; Y), 
and 6,'s are constants related to A and M. 



This choice of TV has residual terms decaying with M very 
slowly. However, our numerical results indicate that this decay 
is fast enough for excellent performance in the short-block- 
length regime. 



We define a pair of random walks to simplify the proofs: 

S n ^i{X n -Y n ) (21) 
S n ^i(X n -Y n ). (22) 

For any measurable function / we have the property: 

E[f(X n ; Y n )} = E[f(X n ; Y n ) exp{-5„}]. (23) 

Observe that S„ and S n are sums of i.i.d. r.v.s with positive 
and negative means E[i(X;Y)] = C and E[i(X;Y)} = —L 
(L is the lautum information lfl3l ) respectively. In particular 
{S n — nC} n is a bounded martingale and hence by Doob's 
optional stopping theorem we have for a stopping time r: 

E[S T }= CE[t] . (24) 

These properties are utilized in the following proofs. 

Proof: Following the definition of (fJTJ and ( l22l . we first 
weaken the RCU bound by ( fTTT i and choosing 7 = M: 

E [min {(1, (M - l)F[S n > S n \X n Y n }}] (25) 
<E[cxp{-[S„-logA/] + }] . (26) 

Then from Thm|4]we have: 

AT-l 



I < 



(i-FM) r f- 



^E[exp{-[S„-logM]+}] . (27) 



where £„ is the same as ( flOb . The proof is in the appendix. 



Consider an auxiliary stopping time r' w.r.t. the filtration T n = 

a{X n ,X n ,Y n }: 

t' = inf{n > : S n > log A/} A TV . (28) 
Denoting E[X;75'] = E[X1e] where 1 E is the indicator 
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function of the set E, we have: 

IV- 1 

£E[cxp{-[S n -logM] + }] 



(29) 



n=0 



N-I-t' 



t'+ exp{-[S k+T , -log M]+};t' <N 



k=0 



+ NP[t' > N] . (30) 
On {V < N} we have i{X T ';Y T ') > log M and hence: 

[S k+T , - log M] + = [S fc+T , - S T , + S r , - log A/] + (31) 



> 



i(X k+T ';Y k+T ')- i(X T ';Y T ') 

k. \rk\l + 



= [i(X k ;Y k )] 



(32) 
(33) 



where the last equality is true almost surely by the strong 
Markov property of random walks. It then follows that: 

N-l 

]TE[cx P {-[S„-logM] + }] 



< E 



N-1-t' 



t'+ J2 cx P {-[s k } + y,T' <n 



k=0 



+ NP[t' > N] 



(34) 



Observe that S n and S n are sums of i.i.d. r.v.s with positive 
and negative means respectively. Thus by Chernoff inequality 
we have that: 



E 



,-[i(X k ;Y k )] + 



' [S k > 0] + P [S k < 0] (35) 



< ai e" 



-ka2 



(36) 



wher first equality follows from (1231 . Thus there is a constant 
03 > such that: 



N-1-t' 



N-l 



J2 E[cxp{-[5 fc ] + }] < ^E[cxp{-[5 fc ] + }] (37) 

fe=l k=l 

< a 3 . (38) 

We assume that i(X n ;Y n ) has bounded jumps, and hence on 
the set {t' < N} there is a constant 04 such that 

i(X T ']Y T ') -log M < a A C . (39) 

Therefore from (l24l we have: 

E[r']<^ + a 4 . (40) 

Letting 05 = 03 + 04 and combining d38l and (l40l > we have: 
j /logM 



e < (1 - p[Gv])- 



V c 



+ NP[t' >N}+ a 5 



(41) 



Let C A = C- A. For a fixed M, we can take N = log M /C A 
for a constant A > such that: 

P[0v] < b 2 exp(-iV&3) • (42) 
Again by noting that S n is a sum of i.i.d. r.v.'s with mean C, 



for NC > log M we have by Chernoff inequality that: 

P[t' > N] = P[S N < log M] (43) 
<6 exp(-6iiV( C-^|^ ] 1 (44) 



bo cxp < —bi A 



log M 



(45) 



Combining d4TT) . d42l and ((45]) we have the following for ^: 



< 



logM i & logM 



+ a 5 )(l-V[( N ]y 



(46) 



Notice that we are only interested in the first two terms of the 
expansion (1 — x)^ 1 = 1 + x + x 2 + . . . on [0, 1). Thus 

logM b a logM b 2 logM 

~ C + CaM^/Ca + C(M^/Ca _ b2 ) + a& 

for some > 0. Hence for M large enough we have (fT9l . ■ 
An expansion of log M t * (£, iV) requires N growing with £. 
The components of the correction term in Thm. however, 
depend on both N (as log M/C a) and M. Indeed for a fixed 
£, all M satisfying dT9b and ( f20b are achievable. The argument 
we make below is that for any fixed constant Co > 0, there 
is an £q that depends logarithmically on c^ 1 such that the 
expansion logM t * > C£ — Co is true for all £ > Iq. We first 
invoke the converse for an (£, M, 00, 1, 0) VFLT code: 

Theorem 6 ( MOV , Thm. 11): Given a stationary DMC with 
capacity C we have the following for an (£, M, 00, 1, 0) VLFT 
code: 



logM/ <£C 



+ 1) + loge. 



(48) 



Combining Thms.|5]and|6]we have the following: 

Corollary 1: For an (£,M,N, 1,0) repeated VLFT code 
with N = (1 + 5)£ and a proper choice of 6 > 0, we have the 
following for a stationary DMC with capacity C: 

logM t *(£, N,I) > IC - 0(1) . (49) 

Proof: We first choose to scale with £ with a factor 5: 

N=(l + S)£. (50) 

Then by the converse we have: 

logM <c | log(l+l) + loge-MC 



iV 



<C-5'. 

The term J' on the right is positive by setting: 

log(i + l) + loge 



5> 



£C 



(52) 



(53) 



Again by Chernoff inequality we have: 

P[r' > N] = P[S N < log M] (54) 

< P[S N -NC < -N5'} (55) 

< 60 cxp {-^(1 + S)b[5'} . (56) 

Since S is chosen such that log M/N is less than capacity, we 

'As opposed to the expression in |10| , we use a minus sign for O(l) term 
to make the penalty clear. 
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also have ( l42l i. By reordering (|4H we have for some b' 2 , b' 3 > 



such that 
log M 

C 



> £ 



1 - b 2 e-^ 1+5 ^ - (1 + S)b' e 



b' 3 , (57) 



which implies logA7 t * > £C — 0(1) for large enough £. ■ 
To conclude the discussion of the penalty associated with fi- 
nite block-length, we comment that N only needs to be scaled 
properly, i.e. (1 + S)£ for S decreasing with £, to obtain the 
infinite-block-length expansion of M£(£, oo) provided in iflOl . 
Thus, the restriction to a finite block-length N does not restrict 
the asymptotic performance if N is selected properly with 
respect to £. However, the constant penalty is indeed different 
for infinite and finite N, which might not be negligible in 
the short-block-length regime. Still, our numerical results in 
Section |IV] indicate that relatively small values of N can yield 
good results for short block-lengths. 

B. Limited, Regularly-Spaced, Decoding Attempts 

This subsection investigates (£, M, N, I, e) VLFT codes 
with N = oo but decoding attempted only at specified, 
regularly-spaced, symbols (7 > 1). The first decoding time 
occurs after m symbols (which could be larger than 7) so that 
the decoding attempts are made at the times nj = ni+(j—l)I. 
The relevant information density process i(X ni ; Y nj ) is on the 
subsequence rij = n\ + (j — 1)7. The main result here is that 
the constant penalty now scales linearly with 7: 

Theorem 7: For an (£, M, N, I, 0) VLFT code with uniform 
increments 7 and N = oo we have the following expansion 
for a stationary DMC with capacity C: 

logM t *(^,oo,7) >£C-0{I). (58) 

Proof: Consider the same random coding scheme as in 
Thm. [4] but now the auxiliary stopping time is given as n T(i = 
n i + ( T o — 1)7 where tq is also a stopping time given as: 

t = mi{j > : S nj = i(X n * ; Y n *) > log M} . (59) 

The rest is similar to the proof of Thm. |4] 



£<ni+lJ2 F [(n 3 } 



i=i 

OO 

< ?i! + 7E[r - 1] 



[cxp{-[5„ 3 -\ogM] + } 



OO 

k=0 



[exp{-[^ o+fc -logM] + } 



< EK] [cxp{-[i(X n *; Y"*)] + }] 

fc=0 
'To J T" I a 3 

logAf 

Ia 3 



< E\n T 

< 



C 



Icla 



(60) 
(61) 

(62) 

(63) 

(64) 
(65) 



where d64| i follows by applying Chernoff inequality and 
is because the jumps of i(X Uj ; Y nj ) is bounded by I x a^C 
for some 04 > 0. Reordering the equations gives the result. ■ 



In view of the theorem, the increment / can grow slowly, 
e.g. / = O(log^) and can still permit an expected rate 
that approaches C without the dispersion penalty. In the 
non- asymptotic regime, however, the penalty might not be 
negligible. Our numerical results in Section [IV] indicate that 
I = [log 2 log 2 M] yields good results for short block-lengths. 

C. Finite Block-Length and Limited Decoding Attempts 

This subsection investigates (I, M, N,I,0) (repeated) 
VLFT codes with both finite N and I > 1. When these two 
limitations are combined, a key parameter is to, the number of 
decoding attempts before the transmission process must start 
from scratch if successful decoding has not yet been achieved. 
The main result follows from combining the results of Sections 
IIII-AI and IIII-BI Once n\, N and I are specified, the value of 
to is implied. Specifically, we have the following theorem: 

Theorem 8: For an {£, M, N, 7, 0) VLFT code with N = 
f2(logM), we have the following for a stationary DMC with 
capacity C: 

I < (1 + PlGvir 1 ^!^ + F[ro > m] + 0(7) (66) 



< 



log M 

C 



0(1), 



(67) 



where t is the stopping time in terms of the number of 
decoding attempts up to and including the first success. 
The proof is similar to Thm. [5] and can be found in the 
appendix. The proof of Thm. [2] now follows: 

Proof of Thm. |2} For an (£,M,N,1, 0) VLFT code pick 
N as follows: 



N = (1 + 5)1, where 8 > 



log(£+ 1) + loge 
£C 



(68) 



The result follows by a similar argument as for Cor. Q] The 
restriction on the initial block-length n\ only makes a constant 
difference. ■ 

IV. Numerical Results 

We give a numerical example of our results for a binary 
symmetric channel (BSC). For a BSC with transition proba- 
bility p we used the RCU bound in ifTol . fl4p[ which gives 
the following expression: 



t 

E 

3=0 



Fig. Q] shows the performance of VLFT codes over a BSC 
with p = 0.0789 with N = 00 N = £ + fl(\og£), and N = 
logM/CA- Since £ scales linearly with OR c , , for the case of 
I + fi(log^) we choose N to scale as: 



N 



logM , f log AT , 



C 



C 



(69) 



where a, b > are constants to be chosen numerically. The 
numerical examples presented here use a = 10, b = 30. We 
choose A = 0.3C and 0.4C, which are about 43% and 67% 

2 We replace (M — 1) by M for simplicity. 
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Binary Symmetric Channel p = 0.0789 



which reveals a considerable performance gap from even the 
most constrained VLFT implementation we implemented. 



0.6 - 



S 0-58 

a 

H 

1 0.56 



H 0.54 



— VLFT with JV = oo, [(181), PPV] 
■- VLFT with JV = I + fi (log I) 
..-VLFT with JV = 1.43 log M/C 
VLFT with JV = 1.67 log M/C 



300 400 500 
Expected Latency 



600 



700 



Fig. 1: Performance comparison of VLFT code achievability 
based on RCU bound with different codebook block-lengths. 
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Binary Symmetric Channel p = 0.0789 



-VLFT with JV = oo, [(181), PPV] 
VLFT with JV = I + fiflogf), I = flog 2 log 2 M] 
VLFT with JV = 1.43 log M/C, I = [log 2 log 2 M] 
VLFT with JV = 1.43 log M/C, 7 = r0.151og 2 M] 
-VLFT with I = N* (ARQ) 




100 200 300 400 500 600 700 
Expected Latency 



800 900 1000 1100 



Fig. 2: Performance comparison of VLFT code achievability 
based on RCU bound with uniform increment and finite-length 
limitations. 



longer, respectively, than the block-length that corresponds 
to capacity. In other words, N = 1.43 log M/C and /V = 
1.67 log M/C respectively. As expected latency increases in 
Fig. [U expected throughput for the finite- N (repeated) VLFT 
codes converges to that of VLFT with N = oo before expected 
latency has reached 200 symbols. The penalty of A = 0.3C 
compared to A = 0.4C is only visible when M is small. 

VLFT codes can have expected throughput higher than the 
original BSC capacity because of the beneficial effect of the 
error-free termination symbol. This effect becomes smaller as 
expected latency increases. 

Fig. |2] shows the performance of the repeated VLFT code 
with various decoding-time increments I. As in ( fl~2b . when I 
grows linearly with log M (i.e. [0.151og 2 M]) then there is 
a constant gap from the 1 = 1 case. However, if / grows as 
[log 2 log 2 M] then the gap from the 1 = 1 case decreases as 
expected latency increases. ARQ performance (in which I = 
N*, an optimized block-length) is also shown in the figure, 



V. Conclusion 

This paper shows that the achievable performance of a 
VLFT code is mostly preserved when the block-length of the 
underlying code is restricted to be finite and decoding attempts 
are limited to regularly spaced decoding times. Specifically, if 
block-length N = £ + Q(log^) and I = 0(\og£) the optimal 
expansion of M t * is achieved. 

The finite-block-length results for VLFT codes suggest that 
it is not necessary to use an infinitely long codebook or even 
a very large one. Numerical results show that a base code rate 
that is 67% of the capacity can closely approach performance 
of a VLFT code with an infinite block-length. Numerical re- 
sults also show that decoding after every log 2 log 2 M symbols 
is almost as good as decoding at every symbol. 



VI. Appendix 

Proof of Thm. Consider a random codebook Cjy = 
{Ci, . . . , Cm} with M codewords of length- N and codeword 
symbols independent and identically distributed according 
to Px- To construct a VLFT code consider the following 
(U, f n ,9n, t): The common random variable 



M times 



UeU = X N x---xX 



/JV 



is distributed as: 



u ~ n p x * 

7 = 1 



(70) 



(71) 



A realization of U corresponds to a deterministic codebook 
{ci, . . . , cm}- Let x(n) denote the nth coordinate of a vector 
x. The sequence (f n ,9n) is defined as 



f n (U, W) = C w (n) 
g n (U,W,Y n ) = arg max i(C w (n);Y n ) 

j=l,...,M 

and the stopping time r is defined as: 

t = mi{n : g n {U, Y n ) = W} A N . 
The nth marginal error event £n is given as: 



(72) 
(73) 



(74) 



(» = < U i{q;Y n )>i{ci-Y n )) . 

Following dT3b-(fT31> we have 

JV-l JV-l JV-l 

E[r] = 51 P[r > n] < ^ P[C„] < £ . (76) 

n— n— n— 

As in fTOl (151)-(153)], the last inequality follows from union 
bound and the fact that a probability measure is upper bounded 
by 1. With a similar bounding technique, the error probability 



(1 - V[Cn})£ < ni + I ]T E[exp{-[5, i3 - logM]+}] 
= E[m + (m - 1)/; r > m] + E[m + (r - 1)/; t < m] + 71 



Vn — 1 — ro 



< [m + (m - 1)1} P[r > m] + 7E[(r - 1); t < m] + IE 



^ exp | - [5 nfc+To - log M] + } ; r < 

fe=0 

^ cxp I - [S„ k+T(j - log M] + 1 ; T < to 



' rn — 1 — To 



fe=0 



< ATP[t > m] + E[n Ta ;t < m] + IE 
logM 



m—1 



exp{-[5„J + };r < 



k=0 



< 



C 



+ NE[t > m} + 0{I). 



(86) 

(87) 

(88) 

(89) 
(90) 



can be upper bounded as: 

V[g T (U, Y T ) + W\ = W\g N (U, Y N ) jt W, r = N] (77) 

N 

(78) 



no 



<Zn- 



(79) 
(80) 



In other words, the error probability is upper bounded by the 
error probability of the base code Cjv- ■ 
Proof of Thm. @- The proof follows from random coding 
and the following modification of the triplet (/„ ,g n ,r) of 
Thm. |3j For k = 1, 2, . . . let (f^g'J be defined as: 

f n (U,W) if n < TV 

f n -kN{U,W) if kN < n < (k + 1)N 

'g n (U,Y n ) if n<N 

gn-kNiU, Yg N+l ) if kN <n<(k + 1)N 

Let the new stopping r' be defined as: 

t' = mf{n : g' n {U, Y n ) = W} . (81) 

The zero-error part is obvious from the definition of the stop- 
ping time t'. As mentioned above, the new encoder/decoder 
sequence {f' nl g' n ) is simply an extension of the VLFT code in 
Thm.[3]by performing an ARQ-like repetition. The expectation 
of t' is thus given as: 




JV-l 



E[r'] = Y, 



no 



JV-l 

< ^P[giP[( N ]E[r'], 



JV 

no 



Eh'] 



n=l 

which implies that: 



JV-l 



E[r'}<(i-ncN}r 1 J2 F ^ 



(82) 
(83) 

(84) 



Applying RCU bound on P[C„] for each n finishes the proof. 



proof of Thm. \Sj; Consider the FV code as in Thm. [4] but 
with an initial-block-length ri\, an uniform increment I and a 
finite to. The finite block-length is given by N — n m where 
n.j = m + (j — 1)1. Define the auxiliary stopping time as: 

t = inf { j > : i{X n > ; Y n > ) > log M} A m . (85) 

Similar to Thm. [4] we have (f86l > to (|90l l, shown at the top of 
the page. Now we are left to choose the scaling of to. Using 
a similar choice as in the proof of Thm. [5] 

logM rii 

~ T 



IC A 



+1 



(91) 



which yields 



N = m + (m - 1)1 > 



log M 
C A 



(92) 



Rest of the proof follows as in the proof of Thm. [5] 



References 

[1] J. Schalkwijk and T. Kailath, "A coding scheme for additive noise 
channel with feedback-I: No bandwidth constraint," IEEE Trans. Inf. 
Theory; vol. IT-12, no.2, pp. 172-182, Apr. 1966. 

[2] J. Schalkwijk, "A coding scheme for additive noise channel with 
feedback— II: Band-limited signals," IEEE Trans. Inf. Theory, vol. IT- 
12, no.2, pp. 183-189, Apr. 1966. 

[3] A. Kramer, "Improving communication reliability by use of an intermit- 
tent feedback channel," IEEE Trans. Inf. Theory, vol. IT-15, no.l, pp. 
52-60, Jan. 1969. 

[4] K. S. Zigangirov, "Upper bounds for the error probability for channels 

with feedback," Probl. Pered. Inform., vol. 6, no.l, pp. 87-92, 1970. 
[5] B. Nakiboglu and R. Gallager, "Error exponents for variable-length 

block codes with feedback and cost constraints," IEEE Trans. Inf. 

Theory; vol. 54, no. 3, pp. 945 -963, Mar. 2008. 
[6] R. G. Gallager, "Variations on a theme by Schalkwijk and Kailath," 

IEEE Trans. Inf. Theory, vol. 56, no.l, pp. 6-17, Jan. 2010. 
[7] T.-Y. Chen, N. Seshadri, and B.-Z. Shen, "Is feedback a performance 

equalizer of classic and modem codes?" in Proc. 2010 Inf. Theory and 

Applications Workshop (ITA), San Diego, CA, USA, Feb. 2010. 
[8] H. Yamamoto and K. Itoh, "Asymptotic performance of a modified 

Schalkwijk-Barron scheme for channels with noiseless feedback," IEEE 

Trans. Inf. Theory, vol. 25, pp. 729-733, Nov. 1979. 
[9] M. V. Burnashev, "Data transmission over a discrete channel with 

feedback, random transmission time," Probl. Inf. Transm, no. 4, pp. 10- 

30, 1976. 

[10] Y. Polyanskiy, H. V. Poor, and S. Verdii, "Feedback in the non- 
asymptotic regime," IEEE Trans. Inf. Theory, vol. 57(8), pp. 4903^1925, 
Aug. 2011. 

[11] S. Verdii and S. Shamai, "Variable-rate channel capacity," IEEE Trans. 
Inf. Theory, vol. 56, no. 6, pp. 2651 -2667, Jun. 2010. 



8 



[12] T.-Y. Chen, A. R. Williamson, and R. D. Wesel, "Rate-compatible 

sphere-packing analysis," draft, 2012. 
[13] D. Palomar and S. Verdu, "Lautum information," IEEE Trans. Inf. 

Theory, vol. 54, no. 3, pp. 964 -975, Mar. 2008. 
[14] Y. Polyanskiy, H. Poor, and S. Verdu, "Channel coding rate in the finite 

blocklength regime," IEEE Trans. Inf. Theory, vol. 56, no. 5, pp. 2307 

-2359, May. 2010. 



